Convergence of the Value Functions of Discounted Infinite Horizon Optimal Control Problems with Low Discount Rates

For autonomous, nonlinear, smooth optimal control systems on n-dimensional manifolds we investigate the relationship between the discounted and the average yield optimal value of infinite horizon problems. It is shown that the value functions of discounted problems converge to the value function of the average yield problem as the discount rate tends to zero, if there exist approximately optimal solutions satisfying some periodicity conditions. In general, the discounted ,value functions cannot be expected to converge, which is shown by a counterexample. A connection to geometric control theory is then made to establish a result of uniform convergence on compact subsets of the interior of control sets, if optimal trajectories do not leave a compact subset of the interior of these control sets.